IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 1 Identifying students with dyslexia in higher education

An increasing number of students with dyslexia enter higher education. As a result, there is a growing need for standardized diagnosis. Previous research has suggested that a small number of tests may suffice to reliably assess students with dyslexia, but these studies were based on post hoc discriminant analysis, which tends to overestimate the percentage of systematic variance, and were limited to the English language (and the Anglo-Saxon education system). Therefore, we repeated the research in a non-English language (Dutch) and we selected variables on the basis of a prediction analysis. The results of our study confirm that it is not necessary to administer a wide range of tests to diagnose dyslexia in (young) adults. Three tests sufficed: Word reading, word spelling, and phonological awareness, in line with the proposal that higher education students with dyslexia continue to have specific problems with reading and writing. We also show that a traditional postdiction analysis selects more variables of importance than the prediction analysis. However, these extra variables explain study-specific variance and do not result in more predictive power of the model. IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 3 Identifying students with dyslexia in higher education A growing group of students with dyslexia enter higher education. Guidance protocols and educational arrangements in primary and secondary education are optimized so that students can get past their difficulties and function according to their talents (Tzouveli, Schmidt, Schneider, Symvonis, & Kollias, 2008). As a result, there is an increasing need for standardized diagnosis of young adults. If institutions want to grant program adjustments and other compensatory measures to students with dyslexia, it is necessary to have objective criteria so that the measures are perceived as fair. Students come with a variety of indications of their learning disability and a subgroup may not have been formally examined at all. Also the students themselves often have questions about their strengths and weaknesses and how to best organize their studies. According to Mapou (2008) manifestations of adult learning disabilities may be subtle and remained manageable in secondary education because pupils developed compensatory strategies to mask their limitations. However, these limitations may become more of an issue in higher education because of the much higher study load. Finally, lecturers in higher education are proponents of valid and reliable assessment as well. It makes them more willing to grant special arrangements like extra exam facilities. Although considerable efforts have been made to develop relevant screening tests and diagnostic tools, there still is much uncertainty about how the screening can be organized efficiently. As a result, there is a lack of standardization across institutes (Johnson, Humphrey, Mellard, Woods, & Swanson, 2010) and there remain many questions about the extent to which findings in English can be extrapolated to other languages (e.g., De Pessemier, & Andries, 2009). To have more information about the specificity of adult dyslexia, Hatcher, Snowling and Griffiths (2002) compared the cognitive and literacy skills of 23 university students with IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 4 dyslexia to those of 50 matched control students. All participants completed 17 tasks assessing reading and writing, processing skills, phonological skills, verbal fluency, verbal abilities, non-verbal abilities and self-reported problems in attention and organization. Hatcher et al. (2002) found that the dyslexic students performed worse on all but the two tasks measuring general cognitive abilities, namely the WAIS vocabulary test (Wechsler, 1997) and the Raven test of non-verbal reasoning (Raven, 1938). In order to determine which tests discriminated best between dyslexics or control students, Hatcher et al. (2002) ran a discriminant analysis. About 95% of the students could be classified correctly on the basis of four tests only: spelling, word reading, verbal short term memory and writing speed. Thus, according to Hatcher et al. (2002) a short assessment with four tests is enough to classify adults with dyslexia. Another interesting study investigating a broad range of skills in students with dyslexia in higher education was the meta-analysis by Swanson and Hsieh (2009) based on 52 published articles. Swanson and Hsieh made a total of 776 comparisons between participants with and without dyslexia. The most important problems of adults with dyslexia were reading, spelling and phonological problems. Furthermore, adults with dyslexia also seemed to have difficulties with verbal long-term memory and arithmetic. Swanson and Hsieh (2009) did not find differences in general intelligence, non-verbal reasoning, cognitive monitoring, visuo-motor skills, verbal and visual perception, social and personal skills, or neuropsychological measures (e.g., EEG). The findings of Hatcher et al. (2002) and Swanson and Hsieh (2009) suggest that a limited number of tests may suffice to reliably assess dyslexia in higher education. However, the data are far from optimal. The number of students in Hatcher et al. (2002) was small and based on a single English university with rather high entrance requirements. Swanson and Hsieh’s (2009) analysis seems more secure to generalize but is also largely limited to the IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 5 English language (only 5% of the studies concerned another language) and is hampered by the fact that data from several different types of studies were aggregated. For instance, some of the perceptual data were based on psychophysical studies of low-level vision and auditory capacities, rather than on measures relevant for educational settings. Indeed, a caveat that must be kept in mind when reading the outcome of a meta-analysis is that it may be comparing apples and oranges given that data from very different sources have been combined (e.g., Sharpe, 1997). Because of the high stakes of correct assessment, one would like to have a broader empirical basis about the number of tests to be administered. The question about which tests are needed for diagnosis further relates to the theoretical issue about whether students with dyslexia in higher education form a homogeneous group or comprise several subgroups. If different types of dyslexia exist, various criteria must be used. Several authors have indeed suggested the existence of subtypes (e.g., Castles and Coltheart, 1993; Lorusso, Facoetti, & Bakker, 2011), either on the basis of differences in performance on a series of tasks or on the basis of a theoretical analysis of the processes involved in successful reading. For instance, van der Schoot, Licht, Horsley, & Sergeant (2002) made a distinction between guessing readers and spelling readers, based upon performance criteria initially developed by Bakker (1981). Another typology was proposed by Castles and Coltheart (1993), who started from the assumption that words can be read aloud in two different ways, either by directly converting letters into sounds or by activating the written word in a mental lexicon and deriving the corresponding phonology. On the basis of this analysis, Castles and Coltheart predicted the existence of dyslexics who would have major problems with the reading of new words and non-words (phonological dyslexia) and dyslexics who would be mostly impaired on the reading of irregular words (surface dyslexia). IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 6 The above distinctions strongly focus on impaired reading, whereas dyslexic students in higher education mainly complain about their spelling difficulties. According to Fink (1998), high-performing adults with dyslexia can be divided in two groups. One group has a slow reading rate and difficulty in spelling, whereas the other group has compensated for their impaired reading and shows deficits in spelling only. Indeed, the writing tests were among the most informative in Hatcher et al. (2002; see also Holmes, & Carruthers, 1998; Shaywitz, Fletcher, Holahan, et al., 1999). Finally, it must be kept in mind that although impaired reading and writing are the center problems of dyslexia, they probably are but the most visible impairments of a larger neuropsychological deficit (Habib, 2000). Neuropsychological research has provided evidence that, although the core deficit in dyslexia is phonological in nature, dyslexic people often present associated problems in a wide range of neurocognitive domains, such as spoken language and auditory temporal processing (e.g., Heath, Hogben, & Clark, 1999), arithmetic (e.g., Perkin, & Croft, 2007), visual attention (e.g., Facoetti, Paganoni, Turatto, Valentina, & Gian, 2000), short term verbal memory and working memory (e.g., Swanson, 1999), timeestimation, automatization, and fine-grained motor skills (e.g., Nicolson, Fawcett, & Dean, 2001). It is not clear to what extent these deficits are a consequence of the difficulty to manipulate speech sounds, or whether they point to a larger variety of cognitive and neuropsychological impairments in individuals with dyslexia, requiring the identification of a more detailed pattern of weaknesses and strengths (Crew & D'Amato, 2010). For the above reasons we felt that a replication of the Hatcher et al. (2002) study was warranted. First, we wanted to see whether indeed a small number of tests sufficed to reliably diagnose higher education students with dyslexia, or whether an adequate assessment required a much wider battery of tests. Second, we wanted to make sure that Hatcher et al.’s (2002) findings applied to contexts outside British education. These involve a different IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 7 language (Dutch) as well as differences in entrance requirements. Indeed, alphabetical languages differ in the degree of complexity in the mappings between spelling and sound (Borgwaldt & Hellwig, 2004; Van den Bosch, Content, Daelemans, & de Gelder, 1994) and countries differ in the entrance requirements they impose on students coming to higher education. As a rule, entrance requirements are lower in systems that see selection as part of the curriculum (as happens in Belgium, with more than half of the students not successfully completing their degree) than in systems based on the master-apprentice model (once admitted, the apprentice is expected to complete successfully, as happens in British higher education). Thirdly, we also wanted to know to what extent it is possible to generalize predictions made by our model (our selection of tests) to future populations. We do not want a model that is only valid for the data upon which it is based; we also want our model to perform well on new data too. An important limitation of the study of Hatcher et al. (2002) is that the results were based on a post hoc discriminant analysis. In such an analysis authors first administer a series of tests and then examine how well the scores allow them to classify the participants. In this type of analysis the more test scores one has the better the prediction becomes, because the test scores are combined in such a way that they optimally account for the pattern of performances observed in the specific group tested. The drawback of this procedure is that it tends to overestimate the percentage of systematic variance, because sample-specific variance (noise) is used for model fitting. As a result, using the same criteria for a new group of participants is likely to result in significantly worse assessment. In the present study we will select variables based on prediction results rather than “postdiction” results (Gauch, 2002). In such an analysis, one examines to what extent it is possible to use the scores of one group of participants (the training data) to predict the performance of another group (the test data). This avoids the problem of model overfitting. IDENTIFYING DYSLEXIA IN HIGHER EDUCATION 8 Both in a predictive and post-hoc model the model fit increases over the first few predictors included. However, whereas in a post-hoc model the fit keeps on increasing (because of overfitting), in a predictive model the fit starts to decrease after a few variables have been entered, a phenomenon which Gauch (2002) called “Ockham’s hill”. The reason for the decrease in performance is that after a certain point the model starts to explain noise in the group tested rather than variables systematically affecting performance. Therefore, the number of significant variables in a predictive model usually is lower than the number in a post-hoc analysis. Models with few parameters may be underfitting reality, but models with additional parameters tend to overfit spurious noise (Gauch, 2002). Following Hatcher et al. (2002), the main aim of the present study was to develop an efficient and optimal diagnostic protocol for the classification of students with dyslexia. By comparing the new data with those of Hatcher et al., we wanted to see to what extent the findings can be generalized to a non-English language. As far as we know, no similar studies have been published.